Field of the Invention
The present invention relates to an information processing apparatus, an information processing method, a program and an information processing system which provide mixed reality and augmented reality.
Description of the Related Art
A mixed reality (hereinafter referred to as MR) technique and an augmented reality (hereinafter referred to as AR) technique are known as techniques for fusing a real world and a virtual world together in real time. The relevant techniques aim at seamlessly fusing together a real space and a virtual space created by a computer.
A video see-through information processing apparatus is available as one of apparatuses for providing MR to a user. The video see-through information processing apparatus displays a composite image in which a virtual object is superimposed on an image acquired by photographing a real world with a video camera, on a displaying device such as a display or the like in real time, so as to present the composite image to the user. As an example of the apparatus like this, a video see-through head-mounted display (hereinafter referred to as an HMD) or the like is known. The video see-through HMD used for the MR measures, each time an image is input from a built-in camera, the position and orientation of the camera in the real space at the time of photographing the image. Then, the HMD draws a computer graphic (hereinafter referred to as CG) based on the measured position and orientation of the camera, and camera intrinsic parameters such as a focal length and the like, and then superimposes an acquired CG image on the image acquired by photographing the real space.
Here, a positioning (aligning) function is an important function for realizing the MR. In order for the user to feel that a virtual object actually exists in the real space, it is necessary to achieve geometric consistency between the virtual object and the real space. That is, from the viewpoint of the user who is experiencing the MR, the virtual object must appear to exist at the position which should exist in the real space. Therefore, the HMD is designed to acquire the position and orientation in the real space of the built-in camera and achieve the geometrical consistency between the virtual object and the real space.
The position and orientation of the camera can be estimated based on the image information transferred from the camera built in the HMD. For example, there has been known a method of photographing, by the camera, an index having its known three-dimensional position in the real space, and then estimating the position and orientation of the camera based on the correspondence between the position of the index in the photographed image and the three-dimensional position of the index in the real space. As the known indices, a square marker, a circular marker, a point marker and the like are used, and these indices are, for example, artificially disposed in the real space. Incidentally, “Ryuhei Tenmoku, Akito Nishigami, Fumihisa Shibata, Asako Kimura and Hideyuki Tamura: Balancing design freedom and constraints in wall posters masquerading as AR tracking markers, Proc. Human-Computer Interaction International, 2009” (hereinafter, referred to as the document 1) discloses the technique of enabling to estimate the position and orientation of a camera by giving (disposing) a feature point marker having the feature capable of being detected easily from a photographed image of the camera as an index, in the environment of the real space.
In the technique disclosed in the document 1, the position and orientation of the camera can be estimated by giving, to the environment, the feature point marker having the feature easily detectable from the photographed image of the camera as the index. However, when the feature point marker is given to the environment, there arises a problem that the landscape of the real space is impaired by the given feature point marker.